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The effects of various questioning strategies on 
reading comprehension were tested. The subjects, 24 second grade 
students reading at or slightly above their grade level, were divided 
into three instructional groups. The strategy group focused on 
integrating textual material with prior knowledge. The question group 
got a steady diet of inferential questions. The control group 
received a traditional mix of literal/inferential probes (about a 
four to one ratio). In general, posttest ANOVA and pre/posttest 
ANCOVA supported all five of the study *s hypotheses: (.1) helping 
children make connections between what they already knew and what was 
in a text increased the likelihood that they would draw inferences 
spontaneously; (2) giving children practice in answering inference 
questions also tended to enhance their ability and inclination to do 
so spontaneously; (3) students trained in an inferencing set, whether 
by suggestion or sheer practice, processed explicit messages from the 
text better than students who focused on those messages: (4) 
treatment effects appeared to be subject to a prior 'knowledge filter; 
and (5) growth in inferencing ability did not necessarily overcome 
the inherent advantage typically attributed to literal questions. 
Although the design ox the study was not ideal and the data supported 
some hypotheses to a greater degree than others, it was concluded 
that instruction and practice asking inferences had direct 
consequences on childres's reading -^prehension. (RL) 
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i 

The Effects of Inference Training and • 
Practice on Young Children's Comprehension 

Several years ago, Guszak (1972) reported a study in which he found 
that children were best at answering the kinds of questions that teachers 
asked most often; teachers tended to ask rather direct .•'literal" comprehen- 
sion questions about four times,.,as often as inferential or InterpreLive. 
questions; students trj Guszak's study performed much bet ter-on literals than 
inferential or interpretive questions* The results of that study have always 
intrigued us because it is pot clear whether the students 1 superior perform- 
ance on literal comprehension probes was due to the fact that such questions 
are inherently easier to deal with than are inferential questions or, alter- 
natively, because students simply get much greater opportunity to practice 
answering such questions. We have always wondered whether it would be 
possible via instruction to alter children's facility to deal with inferential 
comprehension probes/ Further, we wondered whether practice in answering 
a steady diet of inferential questions would be sufficient alteration, or 
would such practice result in only a surface mindset that would not transfer 
to other situations?' Maybe an alternative., approach would be to provide 
training—in contrast to simple practice — in the process of inferencing. 

Until recently, however, we were reluctant to undertake such an 
experimental endeavor, largely because we had no wel 1 -motivated way of 
explaining any conceivable set of results we might obtain and because we 
had few, if any, operational guidelines for distinguishing between literal 
and inferential- question probes. Recent developments in the study of 
cognitive processes involved In text comprehension and i.iferencing and in 
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the technology of question probe development have provided the theoretical 
framework into which such a study might be set* _ . v 

Consequently , we set out to investigate whether or not direct inter- 
vention strategies in the form of alterations i(i;the questioning environment 
in, which students spend their instructional time would result in systematic 
changes in their- a bi 1 ity to es tablish inferential relations hips. 

Our framework is derived metaphorically if not directly from notions 
that have been developed recently to explain comprehension and memory pro- 
cesses (e.g., Schank, 1972; Minsky, 1975; Rumelhart 6 Urtony, 1977; and 
Anderson, 1977)* What has been so appealing within these "schema theories" 
are the procedures that they hypothesize for explaining how new information 
(e.g., that which might come from a text) is meshed with existing knowledge 
(i.e., those knowledge structures that comprise a reader's long term memory) 
in the process of comprehension. 

Of particular interest is the explanation of inference within schema 
theory. The most common kind of inference, slot fill f ng , works like this. 1 

t 

in order for an idea in a text to be understood, if has to instantiate a 

schema (a general knowledge structure) in the reader's long term memory. For 

example, sentence (1) might instantiate a building schema (along with a 

carpenter schema and some affective schemata like pain, etc.). 

(1) The carpenter became angry when he hit his thumb inrtead 
of the nail ♦ * 

Texts are^neve^ an incident (we admit 

» ~ '* 

that our example is particularly sketchy). Authors seldom report what they 
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think readers already know* Notice that in (!) the Instrument for the 
hitting is unspecified. Yet most adults and children would probably respond 
1 •hammer 11 if asked question (2). 

(2) What did the carpenter hit his thumb with? 
This would occur even though the instrument is unspecified in the text 
precisely because when the building 1 schema was instantiated (by cues like 
carpenter and nail) , the default assignment' for the instrument slot within 
the building schema was hammer . Hence, in the act of schema instatlation, 
a lot of excess bagyage (in the form of default assignments of values 
to slots not specified by the text) gets carried along and brought into 
focus or readiness for further process ing. That this is true can be dem- 
onstrated by the puzzlement we would invoke if sentence (1) were followed 
by sentence (3) in the text, 

(3) "I 've just got to get a new rock," he murmured to himself, 
or our disbelief if (1) were followed by (4), 

(h) M l l ve just got to get a new saw, 11 he murmured to himself. 

Default assignments, to variable slots not specified in a text represent 
our best guesses about what should f i t with the schema we have instantiated 
or brought into focus. The practice of assigning default values is 
ubiquitous. We can hardly process a sentence of text without doing so. 
Consider what happens to you when you read sentence (5), from Rumelhart 
(in press). 

- - -{5} - Business had been slow since- the ai J .crisis,. 

9 
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You can hardly resist the temptation to fill the general business schema 
with a particular type of business, such as automobile, fuel, recreation, 
etc. But notice that other cues, in the text, particularly the values that 
fill other variable slots, influence the particular default value that we. 
will assign to an unspecified variable slot. So if sentence (6) follows 
sentence (5) » we fill our business slot with a value different, from that 
we would use if sentence (7)"foTiows^en£ehce~ t5)~. 

(6) Nobody seemed to want anything elegant anymore. 

(7) Nobody seemed to want to travel very far from home anymore. 

(6) predisposes people to fancy cars; (7), to the recreation industry. The 
point is that default assignment is not made independently of the text. 
In fact, knowledge in our long term memory Interacts ,wi th the information 
In the text to tune these assignments; often we find that as we read further 
we must alter our original assignments to resolve a contradiction with some 
new information. 

With respect to adults, we might respond to this account of inference 
with an acknowledgement of its plausibility and some suspicion about its 

> 

importance. After all, on what other basis can adults make inferences savo 
by reference to their existing knowledge? But with children the matter Is 
not so simple. First, we would admit that children's more limited store of 
prior knowledge would make such slot filling Inferences less probable. 
Second, we might question either their ability or inclination to draw such 
inferences at all. In other words, can they? And If they can, do they do 
so spontaneously? 
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That adults can and do draw such inferences in the process of compre- 
hension is well supported in the literature. Kintscn (197*0 developed two 
f versions*of a passage, one complete and the other missing a key item of 
information. Comprehension was checked 20 minutes after the reading, 
latency for answering questions requiring the missing item of information 
did not differ across versions. Kitnsch concluded that the textual information 
had not been stored in memorynHri intact textuaTTorm. Instead Xtw^ 
grated with existing knowledge structures during the encoding process; thus, 
the implied information was as "ready 14 as the explicit information when 
comprehension was checked* Frederiksen (1975) gave adults different sets 
of directions prior to reading passages. He found that subsequent written 
recalls varied as a function of pre reading directions. Both of these 
studies have been interpreted as supporting a constructive view of memory; 
i.e., that interpretation of Incoming data occurs at the point when it is 
encoded into mempry. However, the issue of when inferences are made, at 
encoding or retrieval, is not settled. Certainly the work of Spiro (1977) 
suggests that many inferences can be drawn at retrieval; and the work of 
Bartlett (1932) as well as the Frederiksen study cited earlier (1975) 
suggest that memory for a text is characterized by more intrusions from 
prior knowledge as the time interval between reading and recall increase. 
While there. may be some debate oyer when inferences are made (at encoding or 
retrieval), the common element in all these studies is that inferences are 
an inevitable oart of the comprehension process. 
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If inferenclng Is a necessary process involved in comprehension/ then 
children must possess this skill. Studies about children's inferencing 
skills have suggested that they do differ from adults'. Many educators 
have believed for some time that the difference is qualitative; children 
are not capable of drawing the same types of inferences as adults (Piaget, 
. Inhelder, 6 Szeminsko, 1 960; Bloom, 1956). However, this contention is , 
~7Mrr<mt1y viewed wi th Tome degree of doubt f TheT difference may Be quan- 
titative. Two main avenues of research have led to this tentative conclu- 
sion. However, the two lines of research draw alternative conclusions 
regarding why children make fewer inferences than adults. 

One avenue of research has been pursued by Trabasso and his colleagues 
Trabasso, Nicholas, Qmanson, and Johnson (Note J) proposed a taxonomy of 
Inferences utilized by children In comprehending stories. One of their 
goals was to discover whether the development of inferential ability is 
a function of the content of a child's schemata , Qmanson, Warren, and 
Trabasso (Note 2) found the effect of prior knowledge to be prominent* Two 
groups of civ Idren, ages five and eight, were tested, and it was found that 
when equivalent levels of veridical recall of text occurred, the eight-year- 
old children did draw more inferences than the five-year-olds. Since infer- 
ential limits were not due to memory limits, Omanson et al . hypothesized 
that they were a consequence of insufficient prior knowledge. !n other 
words, young children do not lack the ability to draw inferences, nor do 
they lack memory capacity. What they often lack is prior knowledge which, 
in turn, may 1 imit their abil ity to draw an inference in a particular situation 
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Arfcther series of investigations has been conducted by Paris and his 
associates. Paris and Lindauer (1976) presented seven -/ear -old chlidren 
with sentences such as (8) and (9). 

(8) Our neighbor ^unlocked the door, (implicit instrument version) 

(9) Our neighbor unlocked the door with a key. (explicit instrument 
version) 

-Half of the-ch ? 1 d ren recei ved the---lmp-licit instruraexU-vers ion and half _ 
received the explicit version. Testing consisted of presenting either 
explicit or implicit cues and asking the children to recall the sentences. 
The children who had been given the implicit instrument versions were not 
able to recall the sentences when the cues (e.g., keyj were provided. A 
follow-up study was a successful attempt at "rigging" the setting so that 
the children acted out Implied relationships. For example, upon being 
presented the sentence in the implicit instrument version, the student 
was asked to dramatize the action. However , the word key (which would 
be the cue used later for testing) was not mentioned. Following the 
dramatizations, these children understood the implicit versions as well 
as others understood the explicit. The authors concluded that young chil- 
dren do not spontaneously build semantic relationships even though they are 
capable of doing so. These children did possess the necessary background 
knowledge to draw the required inferences, but they did not spontaneously 
integrate the new information with the old. 

Thus, two possible explanations Ijave been offered to explain the 
quantitative differences between adult and child inferential performance-- 
extent of prior knowledge and spontaneity In drawing inferences. 
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Brown (1977) believes that the skills which are not used spontaneously 
need to be induced in children, tn order for them to use their knowledge, 
theySfcyst employ processes of prediction,, planning, check! t.?, and monitoring. 
According to Brown, research needs Jto focus on the development of programs 
that will train children to apply these processes, it seems that children 

c 

spontaneously draw inferences in their dally activities; they begin analyzing 

-similarities and differences when they-are infants-* — However^they^ia not 

♦ 4 

make these inferences as consistently when confronted with reading tasks. 

The two intervention methods tested in the present study emanated from 
Brown's suggestion that we need to create within children the realization 
that they must and can draw inferences between print and prior knowledge. 

One, method in the present study , the Strategy Method, was specifically 
intended to capitalize on the importance of prior knowledge and to induce 
spontaneity J*y increasing in the children an awareness" that they can make 
infere.ces oetween their previous experiences and the stories they read* 
The method utilized is consistent with Pearson and Johnson's (1978) recommen- 
dation that predictions prior to reading help to highlight the students 1 
related knowledge, thus increasing the likelihood that while reading they 
will consciously try to integrate text and prior knowledge. 

Two recent intervention studies were intended to systematically 
capitalize "n students 1 extral inguistic knowledge as a means of improving 
comprehension. Prior to reading passages, Swaby" (1977) presented sixth 
grade children with a written statement designed to create a connecting 

10 
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link between their possible previous experiences and the text. The procedure 
did not facilitate comprehension nor, specifically, Inferential comprehension. 
Swaby did find, however, that a prereadlng treatment of helping students under- 
stand key concepts In the passage facilitated the post-reading comprehension 
of low ability students. Schachter (1978) added the dimension of discussion 
to the prereadlng activities. His. fifth grade students discussed; questions 

a 

■M h l ch us e d the ward you as the meaaS-Of creatin g conne ct i nj_Ujiiks^ between 
previous experiences and the main ideas of the upcoming stories. He found 
that the procedure enhanced Inferential but not literal comprehension on 
questions from the instructional stories. However, -standardized test scores 
were not affected. 

Thus, only limited support has. been found for improving comprehension 
**y employ I ng\p rereading activities which focus on relating prior knowledge 
to text- _ , . 

The other instructional method utilized in the present study, the Ques- 
tion Method, was intended to induce spontaneity of inferencing b/ providing 
considerable practice in answering inferential questions. Guszak (1972) 
found that reading teachers, when asking discussion questions, asked inferen- 
tial questions only about 15% of the time. One explanation /or the poor per- 
formance of children on Inferential questions In tests -(National Assessment 
of Educational Progress, 1976) may be that they receive insufficient practice 
in such tasks. If considerable practice is provided, the ability to draw in- 
ferences may Improve: Also, if children anticipate questions which require 
integration, they may- develop "a mindset 'toward print which focuses on inter- — 
pretation rather than remembering facts. Such an approach also seems 
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reasonable in the tight of much recent research suggesting that the single 
most potent factor in the instructional environment is engaged time on task 
(Rosenshine, 1978). 

Considerable research has been conducted on the effectiveness of various 
levels of questions upon comprehension. Bloom (1956) delineated a hierarchy 
of question types which has Ueen utilized and modified for nearly 25 years. 
Many CCUCr&la^iojn^s.^ relationship between 

some of these levels of questions and achievement. Medley (1S77) reviewed 
teacher-effectiveness research and concluded that low SES students I- grades 
K-2 benefit from low-level questions, but that no pattern emerged regarding 
high SES chi Idren, 

Winne (1979) reviewed 17 experimental studies concerned with the 
effects of higher cognitive versus factual questions on student achieve- 
ment and concluded that question type makes little difference to student 
achievement. This was considered to be significant f- mse there seems to 
be much professional consensus that teacher questions have a major impact 
on achievement. Andre (1979) concluded that only when the questions lead 
the learner to process the materia 1 in ways she/he would not otherwise 
have done will such questions influence learning. 

thus, it is questicr.il/te whether a method which involves the use of 
higher-level questioning can succeed. However, if the intent is to 
improve inferential ability, then practice in establishing relations should 
be more productive than practice in remembering facts. 

AT though the findings irom various research studies regarding children's 
inferencing ability is mixed, certain predictions seem plausible vi$-&-vis 
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the present experiment. Extrapolating from the work of Brown, Trabasso 
and his colleagues, and Paris and his associates, we reasoned that a pre- . 
reading strategy th«- v focused on making connections between prior knowledge 
and information in print would heighten children's awareness of inferencing 
possibilities, thus increasing the likelihood that they would draw 
inferences spontaneously. Second, arguing from the time on task position, 
we thought that increasing the number of opportunities that students had 
to draw inferences might also increase their spontaneous inferential behavior. 
Third, we thought that this focus on Inferenc'ng present in both treatments 
would result in superior processing of . information expl icitly stated in 

-text. — Ihls^- we r eas on ed, wo uld- occu r because mos t - Inferences- r^equi re an 

integration between prior knowledge and text rather than an exclusive emphasis 
on prior knowledge. Hence the manipulation of. textual information required 
in /inferencing should result In better comprehension of that information 
itself. Fourth, we anticipated that while we could alter children's spon- 
taneous inferencing behavior, we would not be able to eradicate the prior 

n 

knowledge effect (cf. Omanson et al., Note 2). Hence- post treatment transfer 
effects would not be as strong in unfamiliar as they would be in familiar 
material. Fifth, we anticipated that this same growth In inferential behavior, 
relative to a control group receiving a diet of- literal probes, might not 
be sufficient to/wipe out the difference researchers have found between 
performance on literal and inferential comprehension probes. 
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Method 

Subjects 

The subjects were 2*t second grade students attending one elementary 
school in a middle class suburb of St. Paul, Minnesota. Two criteria were 
established prior to selecting the students- (a) the children were all to 
be reading at the same instructional level; and (b) they were all to be readi 

v ± 

at>^or slightly abov€i f jgrade level. Teacher judgment and individual assess- 
ments administered by the school's reading supervisor determined which 2k 
children from a pool of 90 second grade children would participate. The 
students were divided into three instructional groups of eight children each. 
-The ; .experimental cond it ion (Strategy, Question, or Control) was randomly 
assigned. to each intact group. 

Instructional Procedure 

As indicated there were three treatment groups. The Strategy group 

differed from tha other two primarily in the focus on integrating text and 

* m . 

prior knowledge Information prior to reading. The Question group's primary 
focus was a steady diet of inferential questions. The Control group received 
a traditional mix of literal to inferential probes (about 4:1) . 
' Each group met daily with-one of the experimenters as teacher* A tptal 
of ten stories was taught (each over a period of four days), with all groups 
receiving identical vocabulary introduction on Day One (See Table 1). 

Insert Table 1 about here. <|_ 

Day Two consisted of an introduction to the story. For the Question 
and Control groups, suggestions in the teacher's manual were followed. For 
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the Strategy Group Day Two was the day during which. its unique treatment 
was administered. The strategy was based upon a weaving analogy: weaving* 
new information into old information already existing in one's brain.., In 
order to make the analogy graphic for the young children, each child received 
two concrete devices: (a) a piece of gray paper (his/her brain) which was 
slit into three strips and (b) three strips of brightly colored paper to 
represent new knowledge. The experimenter selected three main ideas from 
the story and introduced them one at a time via a two-step procedure: 
First, a question was asked which related to some possible previous experi- 
ences of the children and second, the children were required to hypothesize 
something similar that might ha'ppen in the story. After oral responses 
to the first question, each child recorded his/her own experience on the 
first strip of his/her brain. Then hypotheses pertaining to the story were 

'discussed and each child wrote his/her hypothesis on one of the colored 
strips. Also, at each session the strategy of relating the new to the known 
was reviewed, as an abstract concept. As a follow-up activity the children 

"wove the colored strips into their '•brain." 

Day Three consisted of the guided reading of the stories with the 
focus betng on the difference between the questions asked of the groups* 
The questions asked of the Strategy and Control groups were formulated by: 
(a) tabulating the suggested questions in the teachers' manual to derive 
the percent of 1 i teral /'Inferential probes, and (b) if the questions did, not 
correspond to the prevalent (Guszak, 1 972) ratio of approximately 4:1, 
questions were altered to achieve that proportion. Minor changes were made 
for approximately half of the stories. The Question group received its 
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experimental treatment during this guided reading. All of the questions 
requiring inferences that were asked of the other groups were used, plus 
all of the literal questions used by the other groups were rewritten to 
require inferences. After reading the story, all students completed seatwork 
In the form of a ten-question worksheet (cf. page 17). 

Each of the three types of training questions ?s illustrated with respect 
to a segment of a passage used' in the training. Example (10) is the passage 
segment (Clymer £ Vilscek, 1969, p. 35). (11) is a literal item used by 
the Control group, (12) is an inferential item used by the Question group, 
and (13) represents an item used by the Strategy group. 

(10) Three men came to get their hair cut, but Stanley barked at them, 
the barber looked at William. "Boy," he said. "Isn't that your 
dog?" 

"No," he said. "He just followed me. He lives next door." 
"Well," the barber said, "that dog is keeping people out of my 
shop. There are people here ahead of you, but I'll cut your hair 
now." ... 

"It (your haircut) looks fine," Father said. "You weren't gone 
long. That's what a boy can do when he decides to hurry." 
"It's the barber who hurried most of all," said William. 

(11) What did Stanley do when three men came to get their hair cut? 

■V, < 

(12) Why did Stanley bark at the three men? 

(13) Think of something you did that you didn't tell your parents. 
What didn't William tell his father? 

> * 

Day Four consisted of the skill/phonic activities that were suggested 
In the manual and which followed the curriculum of the school. 
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Instructional Materials 

All groups used the materials normally included in the district program, 
the G inn 360 basal reading program. Workbook pages, worksheets, Magic 
Circle books, and games that were correlated with the stories were used 
with all groupsln order to make certain that the only differences among 
the three groups involved the unique treatmehts administered to each and 
to maximize the ecological validity of the setting in which the treatments 
were administered. 




Testing Materials 

Experimenter designed pretest measures . Prior to the experiment, 
each s tudent read tw o stories and answered 20 comprehension questions 
following each story. There were two kinds of questions: literal and 
inferential. The procedures for. developing and scoring these questions 
were the same as those used for the experimenter designed posttest, and 
are described below in the section dealing with that posttest Jcf. pp. 17*19) 

The pretest was given to further corroborate the equivalence of the 
groups on behavior germane to the treatments. When an ANQVA was con- 
ducted using a correct/ incorrect criterion for scoring responses, no 
significant differences among the groups were found, £.(2,21 ) »,J .986, £> .10 
even so, we were concerned about possible advantages for the Question group 
(M - 20.50) over the Strategy (M - 16.38) and Control (M - 16.38) groups. 
In addition, an AN0VA performed using a second scoring scheme which weighted 
responses to inference-questions on a k ..point scale (cf. p. 19) indicated 
that there were important and statistically significant differences among 
the treatment groups, £(2,21) ■ 3.95, £ < .05, M_ Quest - M.75, 
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M Cont <* 38.38, and M Strat ■ 3^.50. In other words, one of our proposed 
experimental treatment groups was particularly adept at Inferential behavior; 
the other, particularly inept. Consequently analyses of posttest measures 
were run using both ANOVA and analysis of covariance (ANCOVA) techniques, with 
pretest scores and weighted pretest scores as covariates for posttest 
measures. 

Comprehension questions following the instructional stories. Experimenter 
designed worksheets, which consisted of ten questions per story, were analyzed 
for the last five instructional stories. The first five stories were not 
analyzed on the grounds that any treatment effects would not have had time 
to take effect. Each set of ten questions consisted of two explicit questions, 
two inferential, two strategy, and four parallel to each treatment. The 
explicit, inferential, and strategy questions were constant across groups 
but had not been asked during any previous discussions. The four parallel 
questions were different for e^ch group and were repetitions of questions 
asked in previous discussions with that group. They were intended to reinforce 
the instructional treatments. The following directions were given to the 
chi Idren: 

These questions are about the s tory. Some of 

these questions are identical- to questions that our group 
has already answered and some are different. Please, read 
the questions and carefully write the answers. If you need 

help, please ask me. Mrs. .' (the teacher 

supervising the independent work) will not be able to help 
you with these sheets. Please do your very best. 

All responses were scored as either correct or incorrect. An inter-judge 

percentage of agreement of 92 was attained for the scoring. 
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Experimenter-designed post test . An experimenter-designed test was 
administered individually to all the children. Each child read silently 
two expository selections which differed in terms of the familiarity of 
the topic, one being more familiar and the other less. The original 
familiarity differences were made intuitively but these differences were 
confirmed by administering an eight-question prior-knowledge test two weeks 
prior to the reading of the selections. Significant test-scores differences 

0 

favoring the familiar topic were found on the prior-knowledge tests. 

The stories were selected from end-of-first grade basal readers not 

used in the participating school. The revised Spache formula was applied 

to all the stories, and they were judged to be of reasonably equal difficulty. 
-The stories- were altered -when necessary mv orde^ _to~achieve_equal- length 

(approximately 250 words). A primary typewrftbr was used to type them in 

similar formats of 2£ pages each. 

The children read each story in a separate session after being given 

the following oral directions: . 

I have a story about for you to read to 

yourself. Please read it carefully and don't hurry. If 
you don't know some of the words, you may ask me. When you 
are finished, return it to me and I will ask you some 
questions. Please do your very best. 

The experimenter recorded all responses to 20 oral, open-ended compre- 
(tension questions following each story. Ten questions could be answered by 
reiterating information explicitly stated in the text (literal) and ten 
required inference to prior knowledge. The two types of probes were inter- 
spersed to reflect the sequence of the story. The literal questions were 
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generated according to Bormuth's (1969) procedure for generating wh- 
tran** format Ions of story , statements. 

Inference questions were generated using the following paradigm: 
(1) Text segments were identified for which, we felt the text provided no 
useful explanation of the phenomenon in question. (2) Based upon our own 
knowledge of the topic, we generated what we felt was a useful explanation. 
(3) We asked three judge c s to rate the usefulness of these augmentations 

* 

in understanding the selection ( thei r average rating on a scale from 0 to 

5 was 4.0 for all selections). (4) We then used Bormuth's procedure to 

generate a question which tapped the relation between the statement actually 

in the text and our augmentation. It should be added that students never 

actually saw the- augmentations; they were use.d jonly to insure comparabl 1 i ty 

of question generation procedures across literal and inferential items. 

These two item types are illustrated with respect to text (1*0, an 

actual portion from one of the posttests. The italicized portions represent 

the augmentations 4 added to the text in order to create Inference Items. 

Question (15) is a literal item; (16), an inference item. 

Many dogs learn to work for man. A good work dog is not hard 
to train. But a trainer must work with him from the time he is % 
just a pup because if he waits until the dog ,1s older, the dog , 
will have already developed other habits which would have to be 
t unlearned . 

(15) How hard is it to train a good work sdog? 

(16) Why must a trainer work with a dog from the time he is just, a pup? 
The answers were scored using two different sets of criteria.' For 

the first analysis all answers were coded as being either correct or incorrect. 

v * 

20 
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A reliability check yielded an inter-scorer percentage of agreement of 90. 

the split-half reliability of the total test was .743. 

For the second analysis, the following five-point scale was used to 

score the inference questions: 

h - A correct answer: The answer is a reproduction of, or is synonymous 
to, the inserted inference statement. 

3 - A correct answer: The answer is based upon the inserted inference 
statement but fs somewhat broad, specific, or incomplete. It 
relies too heavily on either text or prior knowledge, rather than 
a balanced integration of the two. * 

2 - An incorrect answer: The answer is related to the inserted inference 
statement but totally omits reference to either script or text; 
i.e.,' no inference was drawn. 

1 - An incorrect answer: Such as copying from other parts of the text 
or a "wild guess. 11 

0 * No response. 

On the categorization of responses, the obtained inter^scorer percentage of 
agreement was 39*7. ' 

s/ Free- recall measure . A free-recall measure was administered individually 
at the conclusion of the study. Each student read silently an end-of -grade- 
one basal reader expos i tony selection. These instructions were given orally:* 

*.. r 

1 have a story about ;food for you to read to yourself. . If you 
don't know some of the words, you may ask me. When you are 

» 

finished, return it to me and I will ask you to tell me every- 
thing that you can remember about the story. Also, tell me 
anything that the story made you think of. Please do your very best. 

The students' recall protocols of the story were taped and later transcribed. 
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Before the analysis could be Hone, a template proposition base was 
established for the text (Turner 6 Greene, Note 3). Then scoring was accomp 
Ushed using a "form of recall" analysis in order to assess the prevalence 
of inferences. The forms of recall were: (a) textual information (facts 
repeated from- the text and facts attained by combining parts of the text), 
(b) scriptal (knowledge-based) information (scriptal-textual inferences and 
scripts! elaborations), and (c) . intrusions (any erroneous statements). 
Each student's version was loosely analyzed rather than propositional ly 
segmented. Regarding the categorization of recall segments, the obtained 
agreement between two independent judges was 32%. 

Reading Test of the Stanford Achievement Test . The vocabulary and 
reading comprehension subtests of the Stanford Achievement Test, Primary 
Level I, Form A, were administered to the 2k students as a large group at 
the completion of the experiment. 

• * 

For most, but not all, of the posttest measures three separate analyses 
were conducted: (a) a straightforward analysis of variance (hereafter, 
ANOVA), (b) an analysis of covariance using correct/ incorrect scores from 
the experimenter designed pretest (hereafter, ANCOVA), and (c) an analysis 
of covariance using weighted implicit scores from the pretest (hereafter, 
weighted ANCOVA). 

Comprehension Questions Following Instructional Stories 

The unadjusted and adjusted means for these analyses are reported in 
Table 2. For this dependent measure, separate analyses were conducted for 
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each of the three question types using scores ci the two common questions 
of each type summed across the last five instructional stories. 

Insert Table 2 about here 

The ANOVA revealed a sign if leant effect for treatment on the inference 
questions, £(2,21) -J2.149, £< .01, with the following means: M Strat * 
7.00, M Quest » 5.75, M Cont - 3.50 (see Table 2). The Newman -Keu Is post 
hoc procedure computed at the .05 level of significance revealed no differ- 
ence between the Strategy and Question groups, but each experimental group 
.did exceed the Control. The ANC0VA confirmed the ANOVA results, but contrasts 
following the weighted ANCQVA revealed an additional significant difference 
between the Strategy and the Question group favoring the Strategy group. 
On literal questions, the ANOVA revealed a treatment effect, F(2,2l) ■ 5.523, 
j> < .05, suggesting that the two experimental groups performed better than 
the control group. The Newman-Keuls test confirmed^ his suggestion: The 
two experimental groups performed at comparable levels and each surpassed 
the Control. Both ANCOVAs revealed the same patterns of results. On strategy 
questions, the ANOVA indicated a Strategy > Question > Control ranking, 
£(2,21) ■ 4.365, £ < \05. The results from both ANCOVAs were similar. 

In summary, on measures related to the stories used in the experimental 
treatments, both intervention techniques elicited superior comprehension 
when compared with the Control group, even on the literal measure, which might 
be thought to favor the treatment given to the Control group. In addition, 
where differences existed between the two experimental treatments, they 
tended to favor the Strategy group. 
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Experimenter- Designed Posttests 

Two different sets of scores were created from the experimenter-designed 
posttests: correct/ Incorrect and weighted implicit scores. For the correct/ 
incorrect scores, ANOVA and ANCOVA procedures were employed; for weighted 
implicit scores, ANOVA and weighted ANCOVA procedures. 

Correct/ incorrect scores . For this set of scores, the analysis had three 
factors: level of familiarity and type of comprehension question were 
within-subject factors and treatment was a between- subjects factor. The 
ANOVA revealed no interaction effects. Significant main effects were found 
for familiarity, £0,21) - IO3.667, £ < .001, and question type, f_(l,21) « 
5.579, £, k *°5» but not for treatment, £(2,21) - 1.535, £ > .05. The 
unadjusted means, reported in Table 3, reveal edTpredictable effects for 
familiarity and question type: familiar stories elicited more correct 
responses than unfamiliar; literal questions more than Inferential. With 
respect to treatment, these transfer items did not yield the same pattern 

# 

Insert Table J about here, 

of significant differences as did similar kinds of items based upon stories 
actually used in instruction, although the results are in the same direction. 
The ANCOVA did not alter the interpretation of results in any way and adjusted 
means are not reported. 

* * * 

Weighted- Imp 1 lei t scores . Using the measCire more sensitive to Inferential 
processing, we decided to omit the overall analysis to examine scores 
separately within levels of toplc^famil larity. This decision was made a 
priori on the grounds that any differences due to treatments would be more 
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likely to surface In response to Items accompanying the familiar topic, 
wherein, presumably, prior knowledge wou'd be more likely to Influence 
results* The analysis supported this assumption. According to the ANOVA, 
there were no differences among the three treatments on weighted implicit 
scores for the unfamiliar topic, £(2,21) ■ 1.51^, p_ > .05. The weighted 

« 

ANCOVA results yielded the same conclusion, £(2,20) » .307, p_ > .05 (see 
Table k for unadjusted and adjusted means). However, on the familiar topic, 
both the ANOVA, IF(2,21) • 5.227, £ < .05, and the weighted ANCOVA, £(2,20) 
J*. 157, £ < .05> revealed a treatment effect. Post hoc te?ts (Newman-Keul s) on 

Insert Table k about here. 

the weighted ANCOVA revealed that the Question group was superior to the 
Control group, £ < .05, but not to thfe Strategy group. The Strategy versus 
Control post hoc comparison was marginally significant, £■ .066. 

Unlike the data from the instructional story items, few qpmparisons 
on these transfer Items revealed an advantage for the experimental treatments. 
The comparison that did was embedded in contexts (weighted-implicit scoring 
and a familiar story) In which everything was stacked In favor of the experl- 
mental treatments, Indicating a generally weak transfer effect. 

Free-Recal 1 

On the free-recall data one-way ANOVAs and both ANC0VAS were computed 
on each of the three forms of recall wi'.h treatment groups serving as the 
single independent variable. The ANJV£ indicated that there were no slgnifl- 
cant effects among the protocols of the three treatments on any of the 
recall categories: not for textual information, £(2,21) - .212, £ > .05; 
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•scripts Information, £(2,21) - .*f23, £> .05; or Intrusions, JF(2,*I) - 1.284, 
p> .05 (see Table 5). All ANCOVA results confirmed the ANOVA. The experl- 
mental instruction did not faci 1 i tate^recall of either explicitly stated or 
Inferable information* 

Insert Table 5 about here* 



Reading Test of the Stanford Achievement Test 

One-way ANOVAs were performed on the raw scores from the reading compre- 
henston and vocabulary subtests of the Stanford^chievement Test. In each 
case, treatment served as the single Independent variable. Also, both 
ANCOVAs were employed. 

For the reading comprehension scores, ANOVA results indicated a signifi- 
cant difference among the- treatments . £(2,21) - 7-773, £< -01 (see Table 6). 

* 

Insert Table b scut here. 

The Newman-Keuls post hoc procedure computed at the .05 level of significance 
revealed that the performance of the Question group exceeded that of the other 
two groups, which did not differ from one another. However, the same post hoc 
procedure applied to the ANCOVA results indicated that the Strategy group also 
surpassed the Control group. Ironical ly, the weighted ANCOVA supported the 
ANOVAh (although the Strategy /Control comparison was very close to reaching the 
value necessary to achieve significance at the .05 level). Thus, on this 
standardized comprehension test, the children receiving the Question ^inter- 
vent I pn performed better than those in the other groups,, and there is some 
reason to believe that the Strategy group exceeded the Control group. 
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ANOVA results Indicated no differences on the vocabulary scores among 

f * 

the Instructional groups, F(2,2l) - .665, P > .05 (see Table 6). Both T 

* mm *mm . . ? 

ANCOVAs supported this analysis. The vocabulary test is not a reading c 
task, but an auditory assessment. Hence, the results tend to confirm an 
underlying equality among the three groups with respect to general world 
knowledge, thus strengthening the conclusion that the comprehension differ- 
ences favoring the experimental groups were due to the intervention methods . 
rather than to any world knowledge or general aptitude advantage. 

At the outset, we made five predictions about the results of this study 
tKat seemed reasonable to us in the light of recent theory ar^d research 
regarding inferential processes. T\,e data support all five of those pre- 
dictions* albeit some to a greater degree than others. 

First, we suggested that a technique which focused on helping children 

make connections between what they already know and what is in a text should 

* 

4 

Increase the likelihood that they would draw inferences spontaneously. 
Clearly this occurred within the context of the instructional stories; 
the Strategy group was superior to the Control group on eve:y comparison 
Involving Inferential measures. When scores were adjusted for significant 
pretest differences (the weighted ANOVA), the Strategy group also exceeded 
the practice only Question group. With respect to the experimenter-designed 
*test, the Strategy group did not fare so well. In only one comparison out 
of four (post hoc tests based on* the weighted ANCOVA for weighted posttest 
scores in the familiar condition) is there any reason to believe that Strategy 
treatment induced transfer, and even that is statistically marginal (£ < .066) 
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The free recall data offer no support for any treatment. However, the results 
from the Stanford test, when the scores were, adjusted for the pretest dis- 
advantage of this group, tend to support its efficacy. It is commonplace 
to argue that achievement tests do not measure ''higher* level 11 comprehension 
tasks; however, the Stanford uses. a preponderance of modified cloze Items. 

V- 

We know of no way to determine which choice fits into, a cloze blank save 
by reference to prior knowledge. Hence we are not surprised that this infer- 
ential treatment exhibited some transfer to the Stanford test. Some contra- 
dictory results/notwithstanding, we believe the weight of evidence supports 
the efficacy of the Strategy treatment; we recognize, however, that the data 
better support an argument emphasizing Its localized rather than Its broadly 
transferable effects. / 

; Our second prediction, based upon the engaged time on task argument, 
was that simply enabling children to practice answering inference questions 
would enhance their ability and inclination to do so spontaneously. Lik§ 
that for the Strategy treatment, the data, while mixed, tend to support 
the efficacy of the Question treatment. On every comparison involving drawing 

inferences from the instructional stories, the Question treatment exceeded 

r 

the Control. And on both adjusted and unadjusted comparisons for two of the 
three transfer tasks, the Question group exceeded the Control, and often 
the Strategy group. We acknowledge the fact that pretest measures tended to 
favor this group; however, their clear advantage even in the face of covariance 

i 

analyses leads us to conclude that the treatment rather than their preexperi- 
mentar advantage in ability accounted for the posttest differences favoring 
them. 
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The third prediction was that students trained in an vinferencing set, 

! * 

.whether by suggestion or sheer practice, would process the explicit message 
of the text better even than those students who were given a direct focus 
on that explicit message. This, we argued, would occur, because inferential 

\ 

tasks, while they may have to be resolved by reference to prior knowledge, 
nonetheless require readers to use the text to acquire cues to direct them 
to particular schemata stored in memory ♦ There are three sources of data 
to evaluate this prediction: the literal questions from the instructional 
stories, the literal questions from the transfer stories, and the recall of 
textual information in the free recall task. On the first of these measures, 
both experimental groups exceeded the Control; on the second and third, 
- there were no significant differences among the groups. Whatever effects 
support this prediction,* then, are highly localized within the context ,in 
which the treatments occurred. On the other hand, we would point out that 
an emphasis on inference never r* 1 *ted in a decrement to the comprehension 
or recall of explicitly stated information. 

Predictions four and five were disclaimers* Four suggested that what- 
ever treatment effects emerged would be subject to a prior knowledge filter. 

ndeed, the posttest resuUs on the transfer stories (the weighted ANCOVA 
for weighted-implicit scores) support the prediction quite directly. For 
the unfamiliar story there were no treatment differences; they emerged only 
on the familiar transfer story. Prediction five indicated that any growth 
in inferencing ability would not overcome the inherent advantage 'typically 

attributed to literal questions. The data from the transfer stories directly 
support thc^t prediction; literal questions were easier. However, we would 
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point out that these comparisons^ are a weak test of the prediction because 
there is no control for comparability of content difficulty across item types. 
Even so, a recent study (Pearson, Hansen, 6 Gordon, in press) finds the same 
advantage even when content difficulty is controlled acioss item types. 

1 In conclusion, three comments deserve emphasis. First, we are more 
impressed with the local than the transfer effects in this study. We wonder 
how distant from an instructional seating we can expect children to 
'"spontaneously" apply learned strategies or behaviors. But repl icat ion of 
these treatments with older ^students-rfor whom greater maturity and experi- 
ence might increase the likelihood of transfer— is necessary befpre the 
transferability issue can be adequately evaluated. Second, we underscore 
the ecological invalidity of our design. We did conduct the study In a 

a. 

school, using whatever instructional materials happened to be scheduled, 
and taking instructional groups the way they came to us; however, the teaching 
was done by one of the experimenters rather than > the classroom teacher, and 
the hon-treatment-related activities for each lesson were tightly controlled 
to mitigate against casual confounding between treatments. Tryouts in less 
controlled environments seem* in order. Third, we believe that the prepon- 
derance of evidence in this study supports the efficacy of the experimental 
treatments. While we think that the treatments deserve larger scale tryouts 
In classroom situations by a variety of teachers working with a variety of 
children and materials, we are encouraged by results which suggest that both-, 
instruction and practice have direct consequences on children's comprehension 
performance. 

30. 
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Footnotes 

'By emphasizing sfot-.f filing inferences we do not mean to imply that 
this is the only kind of inference suggested by schema theory- Other kinds 
are not only possible but essential; they simply do not relate to the scope 
of this investigation. A second kind of inference involves what Trabasso ' 
calls text-connecting (see Trabasso, Nicholas, Omanson, & Johnson, Nqte 1, 
for examples). When a student recognizes that the action in sentence 13 
caused the state described in sentence 15 » he has connected two text segments 
via a causal link* Such inferences are common and essential in text compre-' 
hens ion. As we have discussed elsewhere (Pearson & CampareTl, in press), 
*such inferences require the invocation of the Gricean principle of coopera~ 
tlon between author and reader: No author places two sentences in close 
proximity to one another unless he or she is offering the reader an invitation 
if not a 11 cense , to Infer that the one explains, causes , enables, precedes, 
or embellishes the other. A third kind of inference is involved in the very 
process of schema instantiation. In example (1) in the article, the -4*stan- 
t tat ion of a building schema involves an inference based upon the filling of 

a few variable slots specified in the story — carpenter, hit, nail. 

2- 

In this Instance we violated our general pattern of using both pretest 
scoring measures as covariates. We did so on the grounds that similarity of 
pre-* and posttest measures should prevail as a criterion for adjusting post- 
test scores. " 
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Table 1 

Instructional Procedures Used with the Three Groups 



Strategy Group 



Question Group 



Control Group 



Vocabulary 

Instruction t . 

Pre-reading Activity 

'Experimental 
Procedure 

Guided Reading 

4:1 ratio of 

Literal : Inferential 

Questions 



Phonic/Ski 11 
Activities 



Vocabulary 
Instruction 



Vocabulary 
Instruction 



Pre-reading Activity Pre-reading Activity 



Basal -reader 
Procedure 

Guided Reading 

• 100% Inferential, 
Questions 



JExpejrl mental 
Procedure 



Phonic/Skill 
Activities 



Basal -reader 
Procedure 

Guided Reading 

ki\ ratio of 
' Literal: Inferential 
Questions 



Phonic/Skill 
Activities 
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Table 2 

a 

Means and Standard Deviations for Comprehension Questions 
Following Instructional Stories 



Question 



Question Group 



Strategy Group 



Control Group 



/ Type 












SD b 






M 3 


SD b 


M C 


























7 




Literal 


5.75- 


1.16 


5.62 


5. 40 


7.00 


2.00 


7.10 


7.29 


3.50 


.93 


3.53 


3.55 


Inferential 


8.00 


.93 


7.93 


7.75 


8.13 


1.25 


8.18 


8.34 


6.50 


1.07 


6.52 


6.54 


Strategy 


7.13 


2.17 


6.93 

0 


6.72 


8.63 


1.06 


8.77 


8.97 


6.00 


1.93 


6.04 


6.06 



a ■ V ■ 
Reports unadjusted means. 

^Standard deviations .for unadjusted means. 
c 

Means adjusted for total scores of experimenter-designed pretest. 
^Means adjusted for weighted*-! mpli cl t scores of experimenter-designed pretest 
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Table 3 

Means and Standard Deviations for Experimenter-Designed Post tests 



Story: Question Type 



Group 



Marginals 



Question 



Strategy 



Control 



M 



SD 



M 



SD 



M 



SD 



Familiarity Comprehension 



Unfamiliar: Literal 5.38 2.20 

Unfamiliar: Inferential 4.38 1.41 

Familiar: Literal 7.75 1.49 

Familiar : Inferential 7.36 1.41 



Total 



24.88 



4.38 
3.63 
7.13 
6.5a 

21.63 



1.69 

1.30 
1.46 
1.41- 



5.13 
3.75 
6.63 

-5^ 



' 21.13 



2.42 
2.12 
K77 



Low 
26.65 



Hi 
41.02 



Literal 
36.40 



Inferential 
31.27 



ON 



Table 4 

i* 

Means and StandaVd Deviations for Weighted- Imp 1 ici t Scores 
of Experimenter- Designed Posttest 



Story 


Question G 


roup 


Strategy Group 


« Control Group 


M a SD b 


M C 


M a SD b M C 


M a SD b M c 


Fami 1 i ar 


29.50 3.21 


28.95 


26.63 3.34 27.09 


22.88 5.41 22.96 


Unf ami 1 i ar 


23.75 4.33 


22.46 


19.38 A. 60 20.47 


20.25 6.71 20.44 



Reports unadjusted means 

Standard deviations for unadjusted means 

Means adjusted for wef ghted-tmpl f ci t scores of experimenter-designed pretest 



Table 5 

and Standard Deviations for Categories of Free-Recall Statements 



Recall 
Category 




Question Group 






Strategy Group 






Control Group 


i 




SO 








SD b M c 






SD b M C 


M d 


Textual 


3.50 


2.00 


3.59 


3.50 


3.00 


1.20 2.93 


3.00 


3.00 


2.00 2.98 


3.00 


Scrlptal 


2.50 


1.85 


1.96 


1.59 


1.63 


3-16 2.04 


2.40 


1.25 


3.15 1.38 


1.39 


Intrusions 

V 


1.50 


2.14 


1.38 


1.36 


1.38 


1.51 1.47 


1.50 


.38 


.52 .40 


.40 



a_ 

Reports unadjusted means 
b ' 
Standard deviations for unadjusted means. 

^Means adjusted for total scores of experimenter-designed pretest 

Means adjusted for weighted-Implicit scores of experimenter-designed pretest. 
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Table 6 

Means and Standard Deviations for Stanford heading Test 



Subtest 




Question Group 




Strategy Group 




Control Group 


« a 


SD D M c M d 


M a 


SD b M C M d 




SD b M C tf~ 


Comprehension 


85.00 


1.77 83.93 84.19 


80.50 


3.59 31.32 81.19 


77.50. 


5.29 77.75 77-52 


Vocabulary 


3K25 


2. A3 31.21 30.58 


30.38 


2.97 30.41 30.95 


29.50 


3.59 29.51 29.60 



'Reports unadjusted means 

Standard deviations for unadjusted means 

'Means adjusted for total scores of experimenter-designed pretest 
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